1. Field of the Invention
This invention relates to storage systems and more particularly relates to integrating a flash-based cache into large storage systems.
2. Description of the Related Art
The advent of flash-based solid-stage drives (“SDDs”) is revolutionizing primary storage computer architecture, ranging from notebooks to enterprise storage systems. Flash-based SSDs provide random I/O performance and access latencies that are orders of magnitude better than that of rotating Hard-disk Drives (“HDDs”). Additionally, flash-based SSDs have significantly reduced power consumption.
Flash-based memory extension cards are emerging as tools for memory expansion and improvement, even in network environments. Typically, these extension cards are placed directly into hosts using PCI-e slots, thereby placing the cards closer to the processor of the host than to the storage disks accessed by the host via a network. Placing the extension cards closer to the processor prevents additional latencies from disk array controllers and network protocols.
Even though flash-based SSDs and flash-based memory extension cards provide random I/O performance and access latency that are orders of magnitude better than that of HDDs, the random read/write I/O performance heavily depends on the architecture of the solid-state memory controller associated with a device. For example, as noted, read and write latencies are typically two orders of magnitude lower for flash-based SSDs than HDDs, but the latency to the DRAM cache in large storage systems, such as enterprise storage systems that use HDDs for storage, is still two orders of magnitude better than flash-based SSDs.
The overall system performance in terms of I/O Operations Per Second (“IOPS”) and latency not only depends on the internal architecture of the solid-state memory controller, but also heavily depends on the overall system architecture, such as how a flash-based memory extension is integrated into a system and what interfaces are provided.
FIG. 1 is a schematic block diagram illustrating one embodiment of a conventional storage system 100 in accordance with the prior art. The system 100 includes a plurality of I/O enclosures 102a-n that provide an interface between one or more hosts 104a-n and one or more storage devices 106a-n. The storage devices 106a-n may be housed in an enclosure such as a disk enclosure that houses one or more HDDs.
An I/O enclosure as used herein refers to a device that houses various I/O adapters and connectivity devices to provide communication between I/O storage unit processors and the I/O adapters. Typically, I/O enclosures 102a-n are installed in pairs to provide redundancy protection. The I/O adapters contained in an I/O enclosure are typically either a host adapter 108a-n or a device adapter 110a-n. An I/O enclosure has a plurality of available slots to which a host adapter 108a-n or a device adapter 110a-n may be attached. However, in many cases an I/O enclosure is left with several open slots for expansion or other uses.
A host adapter 108a-n is an adapter that provides connectivity between a host 104a-n and the I/O enclosure 102a-n. The hosts 104a-n typically connect to the host adapters 108a-n across a network 112 which comprises various devices such as routers, switches, and the like. The hosts 104a-n typically connect to a host adapter 108a-n using network technology such as Fibre Channel, Fibre Connectivity (“FICON”), Enterprise Systems Connection (“ESCON”) or other connectivity protocol as recognized by those of skill in the art. A device adapter 110a-n is a device that provides connectivity between a device, such as a storage device 106a-n and the I/O enclosure 102a-n. In one embodiment, the device adapters 110a-n may be Fibre Channel arbitrated loop (“FC-AL”) device adapters. In a typical embodiment, each device adapter 110a-n may connect to two separate switched networks that provide communication to the storage devices 106a-n. When a device adapter 110a-n connects to a storage device 106a-n, it may then use a switched connection to transfer data through the shortest possible path.
Communication between the various components of the system 100 is facilitated by one or more processor complexes 114a-b. In a common architecture implemented in typical enterprise storage systems, two processor complexes 114a-b are utilized to increase reliability and to improve performance. A processor complex 114a-b is typically connected to each I/O enclosure 102a-n and includes the logic necessary to provide communication between host adapters 108a-n and device adapters 110a-n thereby enabling a host 104a-n connected to a host adapter 108a-n to access a storage device 106a-n via a device adapter 110a-n. A processor complex 114a-n may also be referred to herein as a central electronics complex (“CEC”).
In conventional systems, the processor complex typically includes a volatile memory such as dynamic random access memory (“DRAM”) 116a-b and a non-volatile memory called non-volatile storage (“NVS”) 118a-b that is typically configured as battery-backed DRAM. The memory 116, 118 of the processor complexes are typically used to cache data associated with data requests handled via the system 100.
In operation, read and write requests from the hosts 104a-n are first handled by a host adapter 108a-n, which in turn interacts with a processor complex 114a. The processor complex 114a manages all read and write requests to the logical volumes of the storage devices 106a-n. During write requests, the processor complex 114a may use fast-write, in which the data is cached to the volatile memory 116a of one processor complex 114a and is also written to the persistent memory 118b of the other processor complex 114b. The processor complex 114a may then report the write as complete even before it has actually been written to disk. This provides fast write latency. However, size, time, and power restrictions may limit the effectiveness of the processor complex memories 116, 118 when used as a data cache.
When a host 104a-n performs a read operation, a processor complexes 114a-b fetches the data from the storage devices 106a-n using the high performance switched network architecture. When data is fetched and returned to a host 104a-n, the data may also be cached into the volatile memory 116a-b in case it is required again in a future read request. In one embodiment, the processor complexes 114a-b attempt to anticipate future reads by using algorithms such as sequential pre-fetching in adaptive replacement cache (“SARC”). Data is held in the cache as long as possible using such algorithms. If a cache hit occurs where requested data is already in the cache, then the host 104a-n does not have to wait for it to be fetched from the storage devices 106a-n. However, due to the size, time, and power limitations of the processor complex memories 116, 118, the caching ability of those memories may not be satisfactory in all cases.